[ci] Use static macOS pool for nightly emulator tests - #8206
Conversation
6b119a1 to
74a59dc
Compare
|
Latest nightly test run: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=8118415&view=results |
f3b39b3 to
9041b46
Compare
|
The few test runs I've done here definitely seem more reliable, though the localization test category is still pretty flaky. My intuition is leaning towards the fact that trying to "soft restart" the emulator ~80 times per test run - https://github.com/xamarin/xamarin-android/blob/69b4ab056badcb5d25a5faf39d08baeb1b8b7010/tests/MSBuildDeviceIntegration/Tests/LocalizationTests.cs#L230 increases the likelihood of us eventually losing the connection to the emulator. I'm going to see if there are any pools with attached Android devices that we may be able to test with. |
Attempts to improve the reliability and performance of our nightly emulator tests by using a machine pool containing static mac minis that are more performant than the hosted VMs.
9041b46 to
a8aeb8c
Compare
|
It doesn't seem any agents in the shared pool have an Android device demand defined, this may be the best we can do for now. |
jpobst
left a comment
jpobst
left a comment
There was a problem hiding this comment.
This seems to be a large pool of real mac machines. Can we also look at using them for our PR build tests?
I have been looking at that in #8192. The owners would prefer we use the arm64 machines for PR builds and higher volume jobs, but I've been having performance issues there that I am trying to better understand. It's surprising because the arm64 pool should have m1 and m2 machines and generally be running better... |
3 participants
Attempts to improve the reliability and performance of our nightly
emulator tests by using a machine pool containing static mac minis that
are more performant than the hosted VMs.